{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 训练分类器"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "目前为止，我们以及看到了如何定义网络，计算损失，并更新网络的权重。\n",
    "\n",
    "现在可能会想，"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据呢？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通常来说，当必须处理图像、文本、音频或视频数据时，可以使用python标准库将数据加载到numpy数组里。然后将这个数组转化成`torch.*Tensor`。\n",
    "\n",
    "* 对于图片，有Pillow，OpenCV等包可以使用\n",
    "* 对于音频，有scipy和librosa等包可以使用\n",
    "* 对于文本，不管是原生python的或者是基于Cython的文本，可以使用NLTK和SpaCy\n",
    "\n",
    "特别对于视觉方面，我们创建了一个包，名字叫`torchvision`，其中包含了针对Imagenet、CIFAR10、MNIST等常用数据集的数据加载器（data loaders），还有对图片数据变形的操作，即`torchvision.datasets`和`torch.utils.data.DataLoader`。\n",
    "\n",
    "这提供了极大的便利，可以避免编写样板代码。\n",
    "\n",
    "在这个教程中，我们将使用CIFAR10数据集，它有如下的分类：“飞机”，“汽车”，“鸟”，“猫”，“鹿”，“狗”，“青蛙”，“马”，“船”，“卡车”等。在CIFAR-10里面的图片数据大小是3x32x32，即三通道彩色图，图片大小是32x32像素。\n",
    "\n",
    "![](assets/cifar10.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练一个图片分类器"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们将按顺序做以下步骤：\n",
    "\n",
    "1. 通过`torchvision`加载CIFAR10里面的训练和测试数据集，并对数据进行标准化\n",
    "2. 定义卷积神经网络\n",
    "3. 定义损失函数\n",
    "4. 利用训练数据训练网络\n",
    "5. 利用测试数据测试网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.加载并标准化CIFAR10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用`torchvision`加载CIFAR10超级简单。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "torchvision数据集加载完后的输出是范围在[0, 1]之间的PILImage。我们将其标准化为范围在[-1, 1]之间的张量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Files already downloaded and verified\n",
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "source": [
    "transform = transforms.Compose(\n",
    "    [transforms.ToTensor(),\n",
    "     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
    "\n",
    "trainset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=4, shuffle=True, num_workers=2)\n",
    "\n",
    "testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform)\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=4, shuffle=False, num_workers=2)\n",
    "\n",
    "classes = ('plane', 'car', 'bird', 'cat',\n",
    "           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了好玩，让我们展示一些训练数据的图片。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 640x560 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " deer   dog  frog  bird\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# functions to show an image\n",
    "\n",
    "def imshow(img):\n",
    "    img = img / 2 + 0.5     # unnormalize\n",
    "    npimg = img.numpy()\n",
    "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n",
    "    plt.show()\n",
    "\n",
    "# get some random training images\n",
    "dataiter = iter(trainloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# show images\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "# print labels\n",
    "print(' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](assets/sphx_glr_cifar10_tutorial_001.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.定义卷积神经网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将之前神经网络章节定义的神经网络拿过来，并将其修改成输入为3通道图像（替代原来定义的单通道图像）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(3, 6, 5)\n",
    "        self.pool = nn.MaxPool2d(2, 2)\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
    "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
    "        self.fc2 = nn.Linear(120, 84)\n",
    "        self.fc3 = nn.Linear(84, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.pool(F.relu(self.conv1(x)))\n",
    "        x = self.pool(F.relu(self.conv2(x)))\n",
    "        x = x.view(-1, 16 * 5 * 5)\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "\n",
    "net = Net()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.定义损失函数和优化器"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们使用分类的交叉熵损失和随机梯度下降（使用momentum）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.训练网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "事情开始变得有趣了。我们只需要遍历我们的数据迭代器，并且输入“喂”给网络和优化函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,  2000] loss: 2.203\n",
      "[1,  4000] loss: 1.838\n",
      "[1,  6000] loss: 1.678\n",
      "[1,  8000] loss: 1.564\n",
      "[1, 10000] loss: 1.485\n",
      "[1, 12000] loss: 1.446\n",
      "[2,  2000] loss: 1.394\n",
      "[2,  4000] loss: 1.369\n",
      "[2,  6000] loss: 1.338\n",
      "[2,  8000] loss: 1.291\n",
      "[2, 10000] loss: 1.262\n",
      "[2, 12000] loss: 1.262\n",
      "Finished Training\n"
     ]
    }
   ],
   "source": [
    "for epoch in range(2):  # loop over the dataset multiple times\n",
    "\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(trainloader, 0):\n",
    "        # get the inputs\n",
    "        inputs, labels = data\n",
    "\n",
    "        # zero the parameter gradients\n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        # forward + backward + optimize\n",
    "        outputs = net(inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        # print statistics\n",
    "        running_loss += loss.item()\n",
    "        if i % 2000 == 1999:    # print every 2000 mini-batches\n",
    "            print('[%d, %5d] loss: %.3f' % (epoch + 1, i + 1, running_loss / 2000))\n",
    "            running_loss = 0.0\n",
    "\n",
    "print('Finished Training')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.使用测试数据测试网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们已经在训练集上训练了2遍网络。但是我们需要检查网络是否学到了一些东西。\n",
    "\n",
    "我们将通过预测神经网络输出的标签来检查这个问题，并和正确样本进行（ground-truth）对比。如果预测是正确的，我们将样本添加到正确预测的列表中。\n",
    "\n",
    "ok，第一步。让我们显示测试集中的图像来熟悉一下。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAB6CAYAAACm9QjtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJztvWmUJNd1HvjFkntV1p7VW/WG7kZjSRJIECQAkTQpWSJFDiUfL5Ttc2ZIm+KMZNMe+cijY8nLzNiW7JFsydIPS0ecoXbZojaKFiXuJAgKBAkisRTAbnQ30PtSWV175Rrb/Lj3xb1ZGxoNne5O+319+mTWi8iI9168iLjrd50kSWBhYWFhMXhwb3cHLCwsLCxuDvYBbmFhYTGgsA9wCwsLiwGFfYBbWFhYDCjsA9zCwsJiQGEf4BYWFhYDCv+NHqA+23gbgF8BcAzA8wA+VKtWTr/R41pYWFhY7Iw3JIHXZxt5AH8M4OcAjAH4LIBff+PdsrCwsLB4LThvJJGnPtv4fgA/X6tW7uG/PQCLAB6pVSsn9L4/+/HPOQD2AVi9+e5aWFhY/A+HMoBLP/HR92x6WL9RE8pxACfNH7VqJarPNs4CuAfAiQ377gNw4Q2ez8LCwuJ/ROwHcHFj4xt9gJcAtDe0tQAUt9h3FQCee/YbiKPoDZ7WwsLC4r9/uJ6HBx58FNjGcvFGH+AtAIUNbUUA69v9II4iHJ5y4LnOGzz1rUcUJzh9LcTRXf5A9h8Y/DEMev+BwR/DoPcfGKAxODu7Kd/oA/wkgI+YP9gGfhjKrLIVPHcwH+AGg95/YPDHMOj9BwZ/DIPef2AAxuDs3Lc3+gD/CoCJ+mzjwwB+F8A/A3C6Vq3s+ADXOLDyJwAAJ4nTtmyGuuW48vbp9boAgDAKaJ9sNt0WxfTbJBYbv+OSmcb15FxJUKJtoG2ZbCfd5sGcU44RxSEAIAjp+EHi4yT+Gg6s/AEy3LUwkgnuxg4fXxDzuBx1IXo9GkMU+ZvG7nLferG0NakbaPXE9FS690PQ+NjHPpZ+D8Nw0zkN2p0u/o9/8yv4yEc+gkI+t2n7TtjqeDsi2fCpm1zdRq2uadSuGsfMH322O13803/9q/jID/99FPOi/O3kjN+q32b/X/7lX9607cBf4bmNwrRtYf4aAKDbkTVz+K4jAIDRkTIAIOPJoLIZWnhZ3cbrOey2cfI//kd84O/8Y2R86sdQKcPHkL76/N1Ti3hpaREAMDw8nLZlMvRb36H9HPVACuMeAMDdQpBzlXTXarboGD6tyXw+n27r9egYId+D7U4XP/lv/zN+5Ef/EYqlUrrfL/78/9N3/H0zlfT70OQxAEDBk/u2PDwEAFjryrpuri5wf+l6x2ox+DyIgi/rNu/xI0zdt+kNyE1RrEy23NbpdnHyyhN4y4OPwOdrZcZO5+e53GLtOA4dxHHlHo3jzWZh89tcjvqbddX9ltB3JyvXtrVAbsOvPPXipmNthzcURlirVtoA3g/gHwJYAPC9AD74Ro5pYWFhYXFjeMOJPLVq5RkAD9/s73v8DkkS5Qtl6TMHebu7MG9Jlqz1q4ffqk5GGrtGaojlDeezhOdxk6+O4cQkFSPsqnNG3B36gRHwe46L0COpp6eO34tcPpa8jR2W4vOqbz5LPi5LX1EQqI6EPCQ5RsIihedt/771PG/bbVvt53neDf8m7drrlcDN72I9FoaSEmMjMiXcn0S2GY3IAa0Jj6Ue3+3v/81K4FthqEjSkZvI7dFtUlvca6Vt+Swdt1Sg/Xx1GrN2cmqRFbL0vcfjzeecdH3kfJJ4s2qdmCnyfRmnkexdR2mbPDc51kr1Mmm2Au6PwGivCeS4Lp8sw1KokeoBIOh2eXxu32c+l0M+K/ttRJyIFB96Y3SsjNzTkUcSuJtREnib3GdJ1OR+yPG6Ce0XKMm3w/OrhHL0AtKSXF4f7ZY8W8yaSXhG1rsu0KHjua7ch4nRXHgytcYfhhHvI+d0nM1S/NgYjTlXGObjyzWLzbrOyQCj9SG8XthUegsLC4sBhX2AW1hYWAwo3rAJ5Y0iYRUSiZguEnYeOZGoeHFAKo1XYDMFRH8xmrR2JGRZ9woTUVHiwOvbz6hCAOAkGxxpABx2uCQeqYIJq9SdZAjXF0hNa/ZELVpfJxXMS+S4w3l2ZjnS33KRnG+FHI0zdnvpNjc1l8jYzQiCeHu1X5sEdjIPmG1Jkuy432udYzv0mSvM/n26ptmkzTc0592A5sPXenPEphM2GcSJ+QRi5eh9vdhpLD6bsVxlxsp6dK6MK205l81jZptyQHbbZGrxPOVw8+m6BxGtHSfqpKaQJKS2xJFbMmKDUzYjztrUdKLWmMPzF7EZsNUSM8/C/DwAYHpyTPZnc4mXlXN5fC4zz8qSA5/37/J9GfOn7znpvboV3ES2RdzfSN0HkUNjzg9LPyYOTNNvV5YAAEMtiUjudegZEQ2JaSYeGQUADGdl7s15Xbaz9rpyf5mABzMHlekJ5DN8n6s1Ydax+XSVzTbkddq3/Pj0WV/WbqFQ4E3GDCgmmtiYZ7UMfRMmSiuBW1hYWAwobrsE7kcseXvy9nNZksh56u1uPET8JnS1p4Z/GmoJ1ThlsiK97Dp4NwBgdfk6AOD6gkgqGZ/ewi5EYuqFND3thBJLQ5YaX77Yg5Nlp4wnTpkeSwbrK4tp2+U5liTySrK6ugwA2L+LzjkxrKU0E1ooYzfCRZRsn8GqJd+dnI1asthpv7+UYtdpv5V2wKGWoRJfAtaETr/6KgBgepeEn8XsjJ4ap/nu9Vi6ukmHatq1HX6fZSk7DkVy81h6yigHWobb3IjWUTYj19HxOFRVaVcZ1+e+99JPl+ch7LDTU62nDo+9WJQ1nMYsa/GP57fJIY7PPFNPNwWsCYyVJc4gl2NnvpqCNJSVtVNXhe85iXHmh32fSRz1KVcbEUJCHV3QWo89+UGX7ydPaWMl9kaWiyTJxvWn02296ySN777/bunbPN1zXUfmbYgHttYmR2hejSXHGnlvhH5XunwNRX5u6EdKt0jb/YC3BTJZayW6LrmVlbTNn7kXANAaHUnbYtaqIr5m+VgcoanGH0mbF71+edpK4BYWFhYDCvsAt7CwsBhQ3HYTitGzHX9UWli9DXWGIjuMeqzWZpVzKIpMXKYyMfAxdFzt2/7q9wIAnnnyGwCAK2xKAYAmm0vCSGIxz19qAADOXrrMncggW30Ii808ZiqH6Jw5URN7rP5lhqbStrBDat9C40raVhybAABcWqfsvo5Sh6eHScUrZkStjAJSg3Wy2UbNdSsn5s3GbW+H1xtPba6tl1FZsxzj3V4Xp/XyCqm6c9fJ9FQYFnV4gjMOTdag+XTg7jw+bVLaoYdbIcvmukQdI2MmP5J+ezDOdmrLqLjqwKjPsRzDK9M8OEkgn7Ex1/AVjWRdr6+SqW2oKE47l+fbZEUCgM+Zy8vsvFxcFdNggeOke8oa2Qs4lj6r1wy1RZzpHCrzkcmCzrJ5L/LN/RZjiwREBXUNjGMxEUdeFPJYle3CYRNHx6HrnollLTiTZFprrUnfgrOnqL+OmJlinq6miS9X91c2oHMGPC/BCy+gE/XnGgBAh82hHseI+3JKdHdRH9vXxFQ67NA974xMyvj4vIFrHMMq9yExeQ0ygb77+s2WVgK3sLCwGFDcdgm869KbdqWlMrRYehkbErGhzE4h34SRKQkhjayKZX/j5Gy1ltK2L/8p8a7MLZNEMbcu76/zl2m/81eEctfLkzQeecR14bg+sgD84UlkirTNV3wcOZOh5spYrvcoC2z3vv1pW4edK6++ShL44rLiZNlLxz04JZpAhkPpHBWyJfIXj1e93ZN4Byem+kwpIzZwR/Ttr6TQrRyHER/FhPTpbFHD+TK/IEyYq00aa1vzX7RoNG6OnMXNtlzboSJLnMZRba41bjzq6vVqIjnHhLzJfBvnZdCVa+XCOPM480+FAPquhNoZeA7NRxL10k8TAhiy4z5SoWbrazRvF/Q5fZO5LNLiTJnmzYQMPv/CC+m2N913HwAg1iGOfP68DrHlzMN2izVcX44fsgbo+XQe45QOwhBRuH0YYaRoo2New4mWGdl52NPhhnzekTWeq6npdFuhcoD6k4jzEBwKmUzuSpvaGTquf414VaBCcpt8v3bGy8CffgXtxx5OQz07Sp0osRbYW6OxdzVHTYEzXptyXfwJ0g6cjAqTZL6TYf6ppyT80KG5d1ydyfr6MqMBK4FbWFhYDCzsA9zCwsJiQHHbTSjzbVIbFgNxYj7+F18FANx7TEwR776PnANjHC+uq/oY0hpXqSORcRSpV9TZ8xRnvNgm1SYpjqfbvCF2lo2Lul8YpT71DIUoE9YETozyGPWtPCR9bFwjk8jqknJusIqXL4ip5cISOU8zZVIPG1fPp9uGrq0BAHaVZf8Cq+ZGzd4KzZYmA2MVUpM9xWbe4vRTYsI5G1CTMcWb3+0mS1TbLtZZvTfOzIIi8+lwxtpVZUJpLNH3WDm4AraLtNbI4du4LvN36fJVAMC9Rw8DALqcVRe5rtjO1PlTnlptNTHd1mkCO7g2PTbhxYE4A1022bVXVGGUbotPyRm4BRl7lq9VVhFROQGZzgwhVtxrwfU4Wy91nIr5qNkkU8HcnPSjVB7ic8r1SXjOe+u0X17Fo88vkyO0/qKYVUo56tORw4fTNp/tZ90Wrb+Cr4iXurS2Ina09jp8DYIW3EjMCJugpthQusZ9uRq8Td3LGTZf5c6cBgB0nnki3RY+zIZDRcuacI5Gdk3ujQ5oHoY438LLyf5xiY6fDZikam4dcUJjHp6QZ1DmMptf1mlNZqYlWAEXaZtfFjNnZ57m1ytKW3yMYsM7TITlqizUbEiT44dqDd9EYrGVwC0sLCwGFLddAvdHSApoLci7JMhSSM5iS5Gd9yisp5ylt3+sHB9G4vQ8KcXZ6ZFkM6+8fdfX6O1bHKUwvrEpcSw2Y5KsJlWFOI8dHr0MO534fdfrheis0/4HpiekjyxtN3oiDTssDa0sihRlYq/a/Hb3stLvuVVypl5dEcnmwCRrGDu8oZfbMtChImkFruJlSItTxIaS04GpRWH8M24fjesW7/YtwhOvXaUQy/Fx0mYKeQkZ7HZozMWctO2aIk0qUeJZs0VjLbGk0uso+k8e9DpTmhrHaOg4iF1ZvhLOaLQK6baz6YvQtGyFvCHsVzsZCTynsmGH2Fk8ws4nl8MhASDH85zXAidrSW5HPrMFXuN8MXqrIoEPl2jtjI2Lpnj2Eml5r168lradOvMlAMDSdZI41ztyjFbwEgDAhwoLZMm+evextO0H3v9eAMBeXs/dvIyz02zy7zjslWMSk9U5OMn2EnjGk/Vn6KCNBggIpaqv5MihJTpXeInCbstKm1i7Qufv5SXbMQE9F5xrjbSttIcdkGXWLCF9LHD4anaR5iB7/iISDuULr19N98vyHIartF9usZxuC9qsLRVEg1k+S8EP2YJI4MO7yenKVErpeQCgyxpPqBzfvZvg9rESuIWFhcWA4rZL4He/6a0AgEtPvZy2DY2QBP7WR9+WthU9shP3mmSv0tKlw2xtUSKMa8OVGQDAcy+cluOOkvS39wCFViXKlpZhKTvuLqRtvV686VwA4DkeXnr+eQBAWRGym/JSJWUHu3JtDkA/T4vHUsU4h38tL4n9bmmRvp+9KqFSe6YpRMrPqmyCDfDLoglELD0Hup4c2xZTNr9MFpFhWks2S5zJFjGFRkDXJQRNQonhy4AK5RzlUKwgUMdiqaw4JDZFI4E7HM7lqJCtXMGEW1Fbh/0RoeOip+uybeqbnDPTvwtv3l4Ev3juHPdb5nttldZdFIimc/kyaR9LvAaa62Ifr0yQ1DxUkiQcjxNgTCjl6uISYg6VdZmLp9kRTa1jBqMKS1y4Qv6Ts5fET9Ds0W/zIxzKVhJJzqzEUlbm6up5Sn65cmUubXviib8AANzDvoapUZE42+sk2ZtyZwEnvly4eBFhew3bIZeVsSdGGo+VSswajKvCXtc58W79LW8GAJT9h9JtrTU6V6B4k5wcz40qN5gp0HmbkeGcUf4W5huJoxj44jWsvOlexKzp6CC+NtvlW+t0zlJBxtLhucwNyX0+PkzPnkg9K9Z57YLDGguBYjTkPumI3+Am+IesBG5hYWExoLAPcAsLC4sBxW03oRRHSPU/cFgcKm22KOw/dCRtm2Q1fPnsOQBAoLO3QjJFvPWdfy1t23/4LQCAQ9Vzadszz5LZY2yITBJXGsKF4iekhuZ0MQHWaNbZiQPHRbEMzM8vYKzo6F2oH2wmmZwSLhRTpOD6kphEHM5WHOYQRN9TjgxWoV+5eCltmxojE9HRfSqUaQM+8Zu/LcfnfmSUOjc0TCrg/r27AQDffvZ5fBerqaZsY6LMPGnhB20vMRw1ykxiHGzZHBe9UIaKbJZU04kxFc5oapuqGoOGYwMZOkZHZfcts1N3mak7DTl//blZODoLkB2PExwKdvSIOJgyJltPFy53+wwqfXjiyad4uKqgiHE8t8VRee4aOdrS2pVKHBrjSvUl5dTNpVGYtOMzL5xEnnlOXK6J2VIOSJ+PkSjen2uL5PgOVJhncdiEvzFf0LoKf4QxPUm/y8N03EceqqZtTaZANiaqCxckg/mVV16hsYdSTAMYxeefmk3XKwD4OXHGA0CpJAEBIc9DEKlwVzbraWphh+mfC9PkqFxtyvHnV2jsjgqP7XHNz6x2Bi7TbwyXUk7V7Vw1AQ98DRZCBxnmQdIFYbqcHQyuebvSlrVmaGiKKlt1eB+ZbD3th0zrufK16rP4mbBXFa55E3GEVgK3sLCwGFDcdgncy5Ej4MrcibTtgYeIfL40Im90b40cRhFLAb4qB/XqRXI0vH3skBy4uA8AMFxSVcR9OleBw/byWVXKmt++e/fsTpu+w5JH1jhj+K29tr6Gg3tJwjt2/N50/8VFLt5QloSAKxze5CiH4ugYSa0rLF1q/pBCkX7bXpN+n77AyRXKETUt1A+0f0tCpXpt+p5RSTVrrABkWSK7dvkaojeTBNZJ2NmjJPAcS0JaajUcGJqlb2SctI3UUaTCD01YlKekbTOHWtaIWRo5x4lWlxsSEra4QI6zdpskN1MG75XTryJSjqsuJ9Xsm6HkqP0z+9JtpXStaCft9hL4c6epH8WCaDwJa3zdUK7LCLNKGmddT0m58+t0DTw1V8N5Dm8LAwB5PH9uDh5fU49jzRxfEsNyTS4+EIhzdHHROC91+S/67DEr4lpTpNweh5fOTEko4sQYLR6TKAQAi0vEozIxSv14y5vvS7dd4lDRFU66ixMAPeDU5cW+0n+HhLYEAOArJtDCMN1z66pEms8qS6RY+HxOdnF5TcYq/NHhAi++upfMt6CnGBhZi/Y5zDSjkqky/Ns28+14iBAzY2ioVmWmwE7GaDOraYa1n0yoNAH28GtGw3xkGCxZeldLziSyaQnaed28mVYCt7CwsBhY2Ae4hYWFxYDitptQMnlyqHSU86bb5fqDKkOxWDJOIVLtdb3MIZ/UxF//1f8vbfvAD32MjtGUjLUs1wI0xSEOHd6bbmsskkOqsy5q8K4KxY0LQT5XT+8FOHyEHKx3HRHn68qzVIuwuSZqonHChCoDrc0mjlGunxclEks7MkbqX6gY+D2XxnfpipgWpt+EPnzwr/+N9HuXM/5Kin8lrYydRHjyqRP4rrc9BkPNsLrK/CShxKNn2Knmq/jXhFXRtoqPTmKu88imE+049Xn/TEZneG42w5j41w7zh5QUx8QY89FEnIHZ6wV49umX8WD1QbTWxFRw6fI5AMARdnx7KkvTmH60OWOnkNtVNtMl2lHIsf0FT+Zj38xd1H9Dm3tN1tp1Nv1MT0t9z9wkmXXWl64DvQC93DASjmkeGSP7Qy4nuQwdvhytUEwoeb4PokDWmMfOQFPkJJNVhSXy9P2tNTGJHDuwh47fk7V+9hUa1ysvfwcA8OjD4uCcmaH9L7xAuRg0d0MI4gS9eHs62azqR5ZrwsaJmC0L7LQOFW3vGmeiRmyayY+I6We6xCYt5eyTiu+Kthem5ifznfibH3NdrjdamTmILAcRaBNKxNfF0Pa66pxZY7hRREtdfqZo7iWfTYgRV6Dvq1vL942uS6pNqTcKK4FbWFhYDChuuwTucIZWS0m+HZYgM5oHYYEdVsx3ksFyum33KL0RT5+QrMsrl87Ql5aUMjt/6RwA4MFdlP2594B4Avc0SAJqnhFmwPEcSX/DnMEZJ8DFVWCoNITde0h6X14V6ShgKXtuXrI5Y+PcUKGCLZbAHeZG0K6LkmE3jCWzMuvQfPSuX8N2iAOREFIJRG0fyhrnGEkWw7lhtFnraQXUj3OvnpNzshNz/6EDadvZizSXf/rZL6VtATNA5pnvpJiXa2ay10bKktU3OkJS1IMPigoxNUlS5137aE5dFb5npKheh7SQDn8enNndJ0bv2U3Xag+HSWqGuxaHmvVpJDuILhl2rE9V9qRteXY2Xr8u4Z1Nzgo26XSGfREARqZobe1VobDDIzTO0tgETj7/IvYfvhvLzG4YsUQWqBJlJmSxpRx/vcBoHaItZQ3jZY6ucSYRDanCcz81Jtcgzw65qTHxOpY51G7hwgUAwPlXzqXbdo3T+l+Zo/BKOC788SHESYTA2/4R4ivuD49ZFvOKH2W5QQ7ZxXXhIJm/SvM7Nkzr//57RRPIsPbdVQ7cgB2E2gFv1r8pcuIqx7qRgl12Trqekz4FNRcQ0uOZY6jwYj6GLqrh8/567ZrfZIxmpG90PryrnMDRDqGt28FK4BYWFhYDCvsAt7CwsBhQ3HYTSkoFq9SR3ZOkPml1/MsvUEz2GJPKHx0XlSafYyeOL7HQ841zdPiuZJTtv4vixD0+brEsDqPJaXIwLSyKurrCzkuljdO+U1Pw2bzTUc7GHqvQ7Y4Q9oT841AdpMPZhGFI78+JSXF0OVwrL+vIWHLs5ImS/kw3jU/9t8+n3+OA1GtXxdAOsUO4wIRbn/rM53DkOJlHpibIZDCxW+h1x7lPeUXGtHyCzEuzJ6RuaDsxxSPob1+pt2X+7ZH9YoZ59K01OldJYqxLrIYbDban5jTk2ObWCpnMzNwtLVzDqDILjI7SuOaYPOy6KgpR4IzA6V0yz8WiygHYgDE2mXnKPNDlwhWOknkWF6hPq6tMC6xMfh5n8J2/LIRR5VUyfwwzkVen2U3jv7tsznKUQy/H1eZRkuteYCeg62tuXLonSgU2L6rK7/smaF6KyqHY5Gr3oTLNmGIXh9jkc+Lkq+m2Y8fupi9pPDPNwbWrV5EblZwHYBwa2mlniovEypyxxuRU8/NiGlxeopjzUy98CwBw8vlvpNuOHKGci4NH7knbxibZDKTMD4Y62RT30IYJj+PAfd7fdx3keG50f02NVykco5ykvNi1HzzNXN7CO56waTXsI4vjo6rrrZ8lNworgVtYWFgMKG67BG6ypEaGxME0OsxlplSI0mpCksT1JXoTTg5L10vsgIlckTzOXTkHAJgeE/L3A/wGN+FZ33pGsj8vXyVJfXhIpPIMhzm9dIYcO+SQHALlW5mQQunjOmfAjSoC/pDFyqtzinB+mPrkc6hSsSgSluEPQSCO0KhJfZuubM+F8vSzL6bfCxlyKHa74mDNshPuLQ8TPedq2Mb5yyRJL7AP6f77JNQsyw7IVlek+AxrLrWaOCA7nOmXZWnx6GHJhr3vHpLc9kyKlFYu0rWNVdjoxWuUBdhY4mIW1+fTbU12bi9zabCIpZk///wXkVc8I4aLxWTqBsqhWBylebsfMr6Rke3n0kjSrbb00WMR1fNFI4mYmtTnDN9Y0dtmc3T8yUnJ7B3iNZ7huSoOlZEr0W99vmY6vDLhTL5QhXeOcIilq7IXY6ZN9U32Ylck6xEmYElC0Qoj1mp6KpPQOLSLvDbPX5P1951XSLvrcmk1OC58AEG7hfgGHW9Gas3nZf6O330cAHDkHgnnba2RNP5SnUJyn/32U+m2J75GGuCJ78haP3bPAwCAo3eLVD46RuvNOHd1tqiRpCOjMXpAJh2Dco6aEnDh5jKGJjszUk7POA1n3B59dM2OKQMpzzFNOX2jsBK4hYWFxYDCPsAtLCwsBhS33YRisuN2VSQm29TIi5UzcPc+Us2/zaaRZUcoWxOP1OyRSXEUjpQ5BjMvqvJBNqEMMYXtr33it9JtLT7XalucXy2OwzW+pDgBEALToxl0Fkmda+b0OcnMc/JliUefmyNzwKrKzhwdpQOWWX32lNMpw9lxXuty2jZVou0jeVHQFCknAGD+oopfHycz0L594rS7901HAQCGB6hQAF56jhxF06zWDjkylgbXByyVxQQ1Uab9fuC970zbXHZojYzQfpMTEr++uEhq+NnzMh8ry2TWWV2R7NM1dhYvM23v4qqQLIXskM0wQZFxEi2sNOErAq+RMs3NKDvVxpS5KccmqmxBTFXr7e1rOU5wDLeOrR/i6iqxokPNuDQfFY4Xd1QWapZjlo1pBwDyaTZiiO889wx27Z2Bn91QhUiZUEwMfKspa8dkBKYOTgAJm1NaKzTfl8/JfC9y8PFoQfafZsrdfF7XkOU6lz71xy+Kg3ie61PO7KZ7LoGDNQD7d09hrbvZxGAQK5pYlxde4uo26punYsNHJ4iW9e3vorV75IiY5L7++FcBAGfPyr3RfJbv21XJC6m+iWiSZ2boWJquOWIyNONYTKIYYWBq3ioTRlr/1XzKJlMvVju0jfVDx5ybtRqZeehzYvIzTplhtEnmRmElcAsLC4sBxW2XwI3TrjwmEngYUbdyvoRlHTtEIW7ffoYkq9WMZLjFDklz03vlTf6dExR+9Nhf+Xtp2zeYqL/ZJCkw6ElBh8Y1Exon77R1rmHnc9abeVuOukvYW6BjrMyLtBN6JPlOV8QRGnHoVVtJfJ02SZxNdpaFsUhYQYcy0SoZkfT2DJGk1A2lbaMEfvnUS+n3VXZ0feD7fjRte+97vwcA8NkvfBYAkEGACjv3KlzFvqBC0/JMlDI9IpLYMH83RQgAIGSpxUiamvPl2sskKV1oSChdjwtz+HmhTR0eJqdvhSVCwy2ikWFHdRCE+MqsOcZQAAAgAElEQVSXvoYHH36sz4k5PExjKZfp01MkE+vMRzM3J9e7owoRbESRpc9AOVoLHBI5WhatJk6pjakfBVXnM3VSKekvTkz4mamW7qX0u8b/mSg+jpCvdxhJX1cXaAz6xs2wBL6+Qtre1SuSfTw9TmMZLU2mbS2u9RqrChQhH9E4TvdygQIAuJvrZD5wL312eyE+/kdP4V1v/y68eEJq2W6EplB2ueCC64tWnWEnfqSyFw0dq8tO3aPHxGEec9jt1at/mLYtXaexnu6K1jZ3mfp011Fykt5znxyjMk1OZXNZPB9wHS42oQqERFzj1VzHLQuA9HGybN6eUhbzPOhDpMVTlGjfl+15g7ASuIWFhcWA4rZL4Ib7Y2xSJISQ39YdVySs/BBLEszgd+GiBP+//WEKD+usyxuxOExhe1cvC3fFmVNUjTs01apVdFGT7a7DExL2tbJCks/IEJcLSxwgAvYfuQ9PP38SAFA/cVb68e73AehnUXz1DEnoy6siRZkQxE6bJO8D0yK5FThpY3xcJN/EJ8kg7G0fZtRpCZdMlQs1fPf3fHfaNjFKtulHHnoM9ed+H4889BgKbNcdZk2gPCRSscdFCvyshHcaW6sm2V9ZIrtrmbWlWDGwHL77fgBAZZ8wNi4ukeYyrBJATJVzh8XQjLowxo5qSn31ul0AX0N5fKqPqN8UCrh4lWz3RssBgIBt5ZofpVjaPpGnydrSsCroYJJ6GornZpWTi2JmLTxiEl4AjDJ/iJfR0iV9N2GBUZQg4HO12PfR6Uq/wx7NlROJRpJ0SYItqTJho6OkwRSyZKP2HVkno6y9jQzLmuzxMVqKbbHHtmyXE0vGlOZVZBbPS+xnCTiZzkti3Hf30XS/eRX+ScfS9ny2d6u+ZXlzrG/E2GgnzIGjtLF9MwcBAAcPHkzbnp6j6x2Gctz5xjL3h6TzEydeSLeZRKX9B8i2fubkc9i3lxLNhofF3wNOqOtw0ZBI3XsZ1rh00o4JI9R5PIkjzyNukcOnBSAE3k0UdLihB3h9tvGDAH4GwAyAUwB+rFatfL0+23gfgF8AsA/A1wB8qFatNLY/koWFhYXFXxZe04RSn20cBvCbAP4BgFHQA/vT9dnGAQD/BcCPAJgAcIm3WVhYWFjcAtyIBH4AwMdr1crj/Pfv1GcbvwjgwwCerFUrXwGA+mzjpwBcrs82frRWraxufajNiLnG4Mi4kPg326S2tCJROQzZual1eOolFZrWIlVlqCRcHsy1j/OnJLzuMjt3Hn2U6GQ1Tecw08OO75GwpQuLZCZpdznkCA7gAdniGMpT5OR5cFhqL86zen3u/HMylhaZG5ZX5FwVrlo/klB/DgxJ6F2lzEUQHDGJGArRklJJJQiPcPj4A+n3v/2//DCNLxI1++Uz5EjsBNS23s3AZedhwOrc4rIifYkND4y4S03h7xjiiFpbpZ54c6TqXlH1LE1hjrgjzqESO0xfPS2mrbNMYWrC8MYnZT6Mur/CVekNN8Vn/vwLQF+YGn13XMMLojJ72WGaV9w67fWNbmBBjkMWF67LWF5ZIuehyWIEgNExclbv3k18HD2VtRf0yDQSJzKnq2zmWl+n2+PitYsAb/fYPKVrLxozSV5Vdy9w+GBHrd2YHX+lIQ5LVesky1mIuliAcQh3lNPOVHo3TsRAFe24tEAZsi2uoUl+ag8Xrl7D3r3i7NwIT5kQ0u/qnHB4vvrC68xvnE3bTBbn8LCYd1LnYl+xDmOSo3OtLcl1fJYpmV98/hlg9Aj+7NN/gIkJMp3s2iVj2bX7IJ+Ttk0o0+oUF6R1lKPcXOdQVbYP2cmZhhHqUEQ2XyXK6Z/EG00ur43XfIDzA/or5u/6bOMRUD75OICTar/5+myjBeAIgPpOx4xUymiHb/JeoBa/ib9V9jKP0+qj1EYms2EiHwI1GeZrrJUMtrWZbTru0mVO5zDSi4nTkDf8HUZAxPvr7Nf0XNqrzqmyOhrBeOcTfiJGiaqgEptqH6otMfHDmrCn3x7uqhs05Jukq9LgI65AYua51+uhyw9H18ylZu0ycxSpKiKR2STH7fFxPbZfB+o6hpzOHqub1mwPVVvMi97h1PRIRwNwFI95cBubeJIk/Te++c6fOgbZ3EDaBm7KEcVbpC/HXEFF8z33Uxf1n9P0zfSVxhds+pUZVxypT3MOEzOs1jxPB0K1hM0ZTHFnQGgDjF1VbzPp255K80/vl1Bfb8NNH/ftA8haS++pWNq32s9A00x0eK0hVNWceP3H7hYPcL6/IvVS9Fxz/6nrwrHyib7PDdn7ltvQ15bATe8r3X8zNx5fM/186vZ4LK5+aRjCKv0A53XBE9YfhcLnj6XRkOH1zaOzed1pOFuxZ22H+mzjCIDHAfwnAHcDaNSqlZ9S268A+GCtWvn6xt/+7Mc/NwJguf7tr/ffSBYWFhYWW8LzPNTe8nYAGP2Jj75nZeP2G45Cqc823gbg0wD+c61a+bn6bOOXABQ27FYEsL7pxxtwdJefvk3/6juIXnQ9EvV2ZZ2lOiW1ehwzev48qdvffEqE/AMzB3gfeTOXS6Qm5mJRlT//uc8BAB55hEwo1xckLviuwxTjevS4EB599ZvPAwBOnOQYa8dFqXI/7hldQp5JguKMRCq8dIKiXE6fkcgUhyNpQuXxv+c4RSvMOOcAAMd3yzSaWp89RUjUM5EYSjKdn/gANILcwfT7sXtpDIuKGtdQtHaCHk688CTuedNjgGe86aaCj5K2WXaMtIrHn26f54TJozgyYWFR1NUoZApWJTCNsqe/F4gpYmmRKX8N6ZDKaDSx7xGTZiFJ4MRrCOMcvIzsV+AsR1NtJqsiVHIc75zJS8cLTCcbLUtWn8HLq9THa1ck0qnIZoyjRyTqYmxiio/FlLAq8qXLceahMkW0mGwqn83gwuyL2F+9PyVxK+aoPwWVYWlMIVqyjziCyphhACDPv83zbxNtGuG5z6qLZqhdmx0xxHWYLndxgTKRFxYl2mZ9ndaRIRQDHBTGDgLdeVTvuzfdL2hJNiQAvP2RR9Pv4/vv5b7JPtmEic20BG5MIYYKVkmgbe7H73/yv6Ztj3/1ywCAYkHWQobHN8zUyZqWOstrKwQQjh2Fv3QaHTbjtlRWaYfNfiGzXk1MSp7KvfdSfPldqqbu5CStheGymP9ynAWegPumtMIwtRRJv1vXqYrYN+tPpm1aE98KNxqF8j6Qw/LHa9XK/8vNJwG8X+1TAZlWzrzW8TzXSR/guawpJSVdyWY4TGmLB7jvmTRWZV4xKblKR/HZPuXrHFg2wxhrg6uSJsz+WRX2la75PlWafm9yIKI+7YzV+75Cr5wyqx7gpu8eq+oZVXoq65oEAjmGCUfSxV+9DYkFsWJcy3AShK8ehGbteLGaW95u1LmtHuCAfoBz2agtHuCxKf6qQwB5R727KSHlRsrMtEHlRV8JLLNtY4iV09dm9jNpzjql2Xz3VFvKULdFgoaTqt4bzqf6DwAeF8s18xz6er2yeUDZRN3EJPfIpxyDP9VLyaxdPd8OrwVfvaCkeLS/qd8Oq+M6NNM8wPuOwdfDrCs9LY4yImo4SPps6xuZCc29DchLJvEUgyRzm9/oAzzhl6E+jTFH6QLUqaWRTRiOTrjh747+OzVjybVK+LqlP002P290GKth48yp8M4clxnc6gHumVOpUOmQj9F3b29a9/14zQc4R5v8HoAP16qVP1SbPgXg39VnG+8BmVV+GsCna9XKa0rgGq+eIeL4/UeFDjLvMi1mT6Rnnx0Y4sgQp+cQ1/07flzicL/4+T8DALRWlBQ1Qc6mM5dISpzZJ07PQ3eTJqAX3eH9tH2ZJcQkcbAC4OSpV5DEXCl+SSSsVXa+dpQ2sbpMb/eKcpCcX6C28RmKhV7IqZjkmJ2eSopKfK4FGIvUujGK+dnnvp1+f2GWnKiOUpAM34SfyQIO8GK9Ds832zO8j7qhmX5W038aPpKs6q/LceJeQtvKWclCdbm2ZODJjdHhG06F7SLLXCVBi+OTmyJd9pgG1UltkAn8HIAoQU/5KyLOtmyu0f5FdR2nRpiyVWWQmvtsK1fm+BStkzFVaCN96Kn5WGOJ0EjDuZzcvMYJqOlI90yTlOZ5Ds4BmBobQy7HD11+6Tc70qMOO4iXl4SfZ2GRYq1NvUwAuIdpezMcW99fwID+0tpbl2t5XkqzjyWG2/g0Wk05/soyae6mert50a43m/jSl7+c7vfOtz2IPjjaz8G+q1DWjnHyxfoFZbRCI2goAeL5+jN03iWJN5/g+PaLV6WtzDHsWV7fscpgLg/xC4rXsuM4yOa4EEZO5UG4NP6FJdIYzp2VTOflJZq3+rcV9w3nTczMHE7b9nCBlN176N7fMy3PmxLTVjsFJay42+cmbIcbkcD/CYASgN+ozzZ+Q7X/TwA+CIkDfwLAh153DywsLCwsbgo3EoXyYwB+bIdd7t1h22viuTMkDe+//61pWwx6+zk65Ijf4Ktchml5WezXE+MUQve+9747bXvgzWSn+uQf/XHa5rA9aYSrg+/dIyGAQ2WSXrxQJI/xXTQ9uw9x5lwcY+XcdUzOHMdzz5OUe3VdeaIzZDsd2S12sMkj1NZXCIB1vJe5SMWZa8qEwqacdke4U5o8DWEsEvL3i3AIAHji8S+k31vMzJbNqFJcxeG0H+/6vu/GV7/w6bSwufHIuxktgVM/8jmR4k0YXlax2PklGms+S+PMKT4HY2Zy8trUwTZzFSHT4WzEVGrVRnPe35RqcxwHe2fGMdeYQ6KzEdnnMVKiazZUENU0xya5jCPSsKPCATciYHVZhx0aM0mk1HGTVWjMelrzNTwt7aaMs80MjIZnY2VhCTm23boZE1Yma/7lE98BAJw/dy5tM1nEiVL39+wm++w4M0K2W2KLN9+Xl8T2vMDZs22l4Ro7e8vsvypakMtzX0xNRDTQxtwcrl4RH8JGCTxQRSRMRqUTyiPHZH32x/qwmZPncn1d2enZD3L3MdHWaw+8BQDwzAtS5OGpp4llc5mLgUQq8qWym5gj3/boY/jSc+fxjnd/b1py79x5CTl+6iniUrqf/UnlEcnSnLtGWv3cnHD8mLW7a1rCDQ8dOkjnZz9Sc018kMYcmfFF6u9swQH0WrBcKBYWFhYDCvsAt7CwsBhQ3HYyq1MrpL5cjxQVZ4ZUarenVI7Y1JCjzz27xYbwjsfIAZnPiFp56ACF+Lz/b/7ttO0P/vgzdK5rdNyrK6K8dToUPJOFqLCLbfp+5rw4QkezLk7OB0imyEQzNi3mhLQungoNitncEDui0hvyphXOlMxnFGkXU7o2HUXGZIoZxFrF6le3pqckO+1qmxw6USRqc5nrdJowqrHxDJaZXnVttcn9ElUzNurvVtlhykySKdB1SDJ0fkNEBgAu21CKitzLVE6Pgs3mMTBpkpMVW0SenZEFNme4roO9GAeSCDOKvnXfbiKPMn7KrgqRcxNaT77KnBst07prSbRcilOnqFbqfSpErsAmET0dJgHKRJrMqSxUQ47WbSszBZsEoyhAFsCLLzyPw1ywYKpC/Y/UCTJ8rUYVsVSO+6HLPJoQwJMvE43quioAYbZpqtSYTUTNNZmjFvezxdmiPWXiMsUjLnBxEtf1cHTsIBYWFjYl72gkfRmWJtFKtjtp9Ji0xcbxyZeqoOrFvuNd38ObdEgk9e3YA2KCvf+hh6mfpsiCcuuagiN7983gS8+dx/1vfgQlpl8+eFRoZ/fsJ8dwgTN6R5QJxYxrUYVaGjNJZUrCDQ05lok0cpW3NuKAhEDZ3eLXSNrZClYCt7CwsBhQ3HYJ/OVleof8yddn07YHDpA0sisrBv4iSwG7d9EbbvekSCV3HWZnZCJSw1XmJfnEf/1M2vbMc+QU6jJRv/aRmrjTREmhUY7OEbHE6TrAaBaYDwroGWpQVzn5zGyqlPdOzxD2qxh1dmiamOxEcYWE7NLJqLe1Ka3VC7aPCU0CkdhHSiSlrSlHaBCRVHb8bgpzOnzXHiTT5LhtzJMk3lCJTevMi9JSDjHj6EoiOW7JJynj+JuJpvOKKpU2v0rhl+2eSIRtTnDRMec5prMtsaYxqrg/prjC+O49dN0T/vfOh+7FbpW8sc6hh4scZudlZf6KJXJaDw3LcScmqO3KWXFcGQQsvXfWRYNxjfNQiZCGHsGke58+fSrdtrZiHMkqv4GLXvieh2wui1a3h5hTtl2T2aFCIydYa9LO0RZTELfbMqcXL17q289RYlnCQeStnlwzk5DTvC4arokhNyXsQpUt3eQwwpBDF02MO2VUby81tpX24XFIpK9oI3p8v4ZQ6edhP+2BpkQwAn2oHL0m7r+n4u337Gc+I05Td1S6usv3+fmLS+lnmKz2HQsAhkcO9Z1/aUXOaWL2S+WDMljDJ7QiY74yt8jHoI7nVMw3M9LCGZL10VnavszfdrASuIWFhcWAwj7ALSwsLAYUt92Ess5qxRfron6eeoWyM7//IXEi3bWHVPWzrxKN7Dsfvj/dlmfVe60n6tknP/s0AKD+HakP2DJZYGzCcBV1p1FzXJ3C7hqWMk2+5aMLF11W8RwVW9xFf+V0APDZkadTjotFGnMWhq1Mjh4ZhkLVaFj9ssNSxQbor5uxcEXoWaOAVLG2Um9bF4lDZjJbwPDDd+HqmdOYYNNFhqvAFFTF8LZnKoxoO5OZB3XcNpld3slVke67p5puu3CBzBMLy0tpW8qQqJxfPjumC+x1mlTx16PMZxHB0HXGePzqIjJLl/Dymavpfg4798oVclIVyuLgLA7TMTRN7ZBySm1Ega9ZT5kpjHO5r86jif9m80O5LNnBeY6pHyqJE87jcRWyWazNzWNm/97UZHH6JBF7rixK1uVKsz9uHAAyWc4IVespx/q4Y6rTq2zOBjvaWl1Rzz0ew9iIrKcem9tabTpnGMg549Sc4vR/Oo5QHWyBr30tJTHFSkhVcUq+zEfE6y5QZhLjSI+4CpG+lwJTyUjdj8ZB2OlKW5RWeGKKAVVbd3yUnd0lMo8+/cyziGLD9il9dziF3UnpDDbTO7jKJOL79N11Nu9nhtCX3sDUnk5RUT502Py3M/1JH6wEbmFhYTGguO0S+ASzeC0uyevvKmeNPfl8SjeOKDjA3+hNN7VLsigdj96w3/q2ZGN95suUSdWN5Y0P34SibX5vRSwZJuo1bMLDjBRAfgofURIjw29+R78umcfB14RH7PDR3C2GVMlLWKJQXM2xIb5REvjuXSQtDpeV1Njql8B37R5Pv1+6cInHpMnz6fv5M6dx7OH34vyZ01gylcJ5l6YKU2yytBNHfZ5e2l8R7PS6JLHVv/55AMC7SjLO+3mc7RGRho3TTmfZdtjBtsLZkdqZev4kZbtdb5OjKXY84OCDuHD2RRSnRIIc20Xfc2W63p7KxCxyGF6uKE5xx9t+6ZtQ1SiUa2CyeGPFod3lsRsnZkGRWRmujbbiFOkukjbYajexe3wGp07UEbEEbni7M5rMijXFTF5J/XyKnuLaXlsiibvTWedPcTybK5VXaz5o01oPVP1S43A0n9p5aML9Qp4XU8MyQhaZzPbiYl5lAgceX5dY5ijHQQKxCj1NubNdw8Wv70fmPVdSrslIjROVZWt4/E3dScXoZ4R3l8NwXQRwPFp/hh+fxmzSiOlD19w0/OtaqzZrxumrMt//nOmprNKEj9FR05fzSFvas+cAbhRWArewsLAYUNx2CdxIq5mMYinrkPR0dk74GLpNSq54Z40qnBdGVfX4Dr21H/+mMPK12XYbKDa4HIdxGelCh8gZeEoaSF+maUIAHydTQGKSWVy1P7OaFVQ5LxNyFKjElTWWykwSRFdJeiNjHELJiSkAMMTxiW2VeLHx1bv/mDCdrXJIXfPSdbUHs9KxxNcJAoTM153lMfeUvVvsrpvDxJwtioCcfoH4Jy6uSRjmlEvzoe2YEUsl68refi0hqe8M20QvKQ6NVpE1mP3EYWHoRqeOHkBxTCT79DqwVDQ0JJpAke3hrlpjyQ6221Xm2WmtSRhh4wqtyU5H+hZxPw0Phr7GRpNzVfJQhhPN0pC9XjuVNH2WZHXIYMB2YM2n0u3S2llT4WrmspXKHJ6qJL+Eq1t111W1e+YGWVESp5G8o7Qyjy4XtnWpL9/PAEG05TZAJYMBWG+SH6To6WtAn5FazEFaAYf51EMVWudyGKuStoX1Ue5DU1UrMtqeutYmicn1XPjjh3D27LNIuGZAt6O5YaK+/bVmnqS+IEU/q7h6DMw3cwyvp/vN3DNqDe+eoe97YCVwCwsLi//uYR/gFhYWFgOK225CSZ1C2pHnkSrYU06WuXVSx+ovkyPofS1RadYSMi1cXhITQ55V6LAlxzCFVYtFMnH4KkvObOsLE3NMGJJx9iUAQvheDhH3LZMTmth1Vid7ipLWmFO0GcGYTJqcETo0KuaSMeZS6CkKzJMcYpZR4VMPbdCyymPixJyaJn6Sq8qEkqpz6rPLZpKAuxYplTDaIcOubwsfOGAVvHldiPXdHFP0qhC2K3yO51Rl+zM+z8cQmaVKM1IUYmoPcdpMcJGFKAFeXQOKw0N9fTRFZXNcJcXztSPZtCkn4w6xWtfOUUirrhJuVGpHZ9RyAQdTnVyrz1k212geGLPdmPDiOE5V6fV1Ljbd1ZwlpjqODumjdZFVxQem9+7hY1DG5OqShG2GXKChr8waX7RWT5tVjHnCxLxh0/6ZlLuES9R5Mda0WW8DLl6UoILTV6kfpawKvWPbT9RXgoKrG5nCz8qxns25m9qMySXS1EA8z8bJqKv6GJOV42cwNX4ISyuvptS12nFrwl1jDibQ191lk2Mf55EpWJFszhw1vwwUz1I0Tutib1VCpUfMJX0dlChWArewsLAYUNx2CTyNntdE+Vz+K1a8CYaP5GyD3vif+OSfpdu++11E6n72ikh/TS5WGat3VMYwubEUUFRhQNkCSdLtNZGejXSUsMRML2EHSSISnnZcGUkvVg6Pdmt9U5vZb5Sl5glFAj/PRWWXrwsD4vJ5Sl46cvgQtkMhLxJZjhNGMooPJGJnls7FCNNahKYmoDrgDlJAn0uLpZ11Ht9JJdWNcLm1kx0hvn+JtZOFskimEzM0rt2HSNoeVSGROQ5LdA2fRQxgDQiSGBlfpDmPJV4/TWoRiSmVnnWI1w5OTI+dWrEK5YxZ0us7LjtO3cRIZHKMLodEhoGsJyNRIwEO7DmO+bm51GlonN2ZrDj5PGYj9LUTmNdiPqfC8Qr0m8UFOqdmGTS1MD013h5rm6GWFjc44foSV0yBC7+/pme3tYZWc1Oh9BRuosrxGWk0EqnVSPt9yUBcmNxNTKie0qRY8lV5TencJ7r4L48hkZjBFEbKzrCk7yKD0BQZUU78mJ9BiWskazlGymukOuJg81gSTggLOWGwvEeYCvdVKRjDd+R6L59iPqh907hRWAncwsLCYkBhH+AWFhYWA4rbbkKZ4EraHUXA3+RMsawn8dSGZtLE8j7+rRfSbWevkGNzuSmejMV1UoOVLxAlVsdDVqNyqrq6Ub3zBcWzwOqnzwUXSE1aQbGUR5BSVWpHGnObBNKPHsepFlQ1c0MqPz5JppOecuB2uYBBOyfmgZiz83TF8o0IVMZkk/kshkflnJ0mO2nZY5PNO0hCzqgzQ1BUps724b1phhsAJJzR2OQY3SdUEY7zLWpbUHwP/jRV6N69byptOzRF3ydGaF5clc3ZZNW0w2qusaREGRclxZmS5yxLn6uD5wtiosnx3Ossx50Qb8HDYThhEkVNmrD3NzXRqGOYTL5ImwB4HeW4zmg2l0t/Y8wSfVas2KwncQJH7CzuZeTamgr1xnQSa4clc6d0VM5DmlmsaWfN/sb8oPrh81iSHq2hxKN+LS/No72DCSVUceAR19/sqfh/E48OlZ1pwvljvpdcdQ0MZWysTR2RcQgrcyg/I4w1Q+9vTGBmm+MCsYnPViYzYzZKs6p1fDebeaAdrHzAQD0PAqZ1Hr/7LgDA3oMz6bYO19N85aTkrhQC5t6RJPPXhJXALSwsLAYUt10C77BUmVOvki5LQBlPpFAWFlOCercgUto5dl66KnQsZOkoVFmOHWZca3ImpHbUGGm8pCqdF9ix6bLU4CQJkADFQgZ5ZjPTnBTzzCQXq3Ahnx0YY2VxMu4aJ61j1y5y1i03RVJZZea+9RXJAhxlYv/r8zqzchIagaqy7mVp7GNTcs5giOYyYafP+MQQOhxiZtjgEiWBu6nPU1eUZ+lMe+uMo43Z+gLFQdIdoX7fNSpOmbFxmrehsiy9oSJdtxxnnHYU30iPww4TU1bOjDHj93sN+XuGNSkdRmgyH7VDLNnBS2vmxVdhh2bMfaGIPHZT2EGvJ6O99YUrmnBGz08/DY+KCeOLYlk7Ac+DpzSvgPk0IhXuWuqS5mIkb81V022z9L5F6bN4i4xa0w9faStm3hbniH/HOFeDbhMOts7SpB3V1ww7R7NyzoxhA436KlDwT5lbRB0uMQx+SgPMs4YxVhbHtymhZgqQ6Dn1jAaayyMCMDU6LhmbOvuU7wnjZF5bVTwmvDxjX+ZohakG/Unpx4Fj5Kgc4+zqyyfPpNuunyHGVV/1LZ/ZvmDLdrASuIWFhcWAwj7ALSwsLAYUt92EYlS8nCL9KRpHRiCqownzjFll0wQ7salP2VNOp8hQSmpHFDvCUspKeX8tLZLpYlGds8yFAEY4XtsFUBny0At6yMGoZ2K68FnF83KiO3aZ/MgUBND7hS2uNdhSpD/LCzx28b7mOeOvs0P2oKfUr9EJMi8NlVQceNc43OgYlV0TCAJ2SsHE/ipiLn63u330mGwW8JVTklXiIpsshodVhuAQmYqGcuKMLmWNA0/Uzx5/Xee49bZyyEbsHMqzumr8TFnXS80lgJgqjPNQX/ceO6myWeV02oEG1WTXuspMkTGmO00UxX1LQ87DFdQAAAdCSURBVOu1RSLNzFN0vCY2nOukhq01hD0TL86ZmD257m02nURtlTHJTsySMjMVRib5uHSMoCPH0NfPII2H15mmadF4+lJSORJNrm26ukpmPePURyJZiVvBC9Uc9/i+VRm4CVO6elAUuvxdslaVA9JJ+j4BIGYTVMsX4jvJpjZ00OpZwdnSnbCL0giw2lxJHZZOXyx52kk+kkr15PNrB3W5Qs+IqWOSq+Hys+rlp79J52yICdTj66cLc2xl0notWAncwsLCYkDhJDfx1L8Z/OzHPzcCYLn+7a/j6LQLz339BvvbjShOcPJKgON7MgPZf2DwxzDo/QcGfwyD3n9ggMbgeCjueSsAjP7ER9+zKWbTSuAWFhYWAwr7ALewsLAYUNgHuIWFhcWA4pZHobieR/mrW3jH73g4CTwvppCYQew/MPhjGPT+A4M/hkHvPzA4Y3C2j5YCbu0DvAwADzz46C085V8+aq+Dp+BOxaCPYdD7Dwz+GAa9/8DAjaEMYJMT81Y+wC8B2A9g9bV2tLCwsLBIUQY9PzfhloURWlhYWFj85cI6MS0sLCwGFPYBbmFhYTGgsA9wCwsLiwHFLXNi1mcbbwPwKwCOAXgewIdq1crpW3X+m0F9tvGDAH4GwAyAUwB+rFatfL0+23gfgF8A1c74GmgsjdvX051Rn23cC6AO4P5atXJmkK5FfbZxENTX7wKwAOBf1KqV367PNo4C+ASAGoBXAHy0Vq1887Z1dBvUZxvvAPBLAO4CcA7AT9Sqlc8OwjWozzb+FoB/XKtW3sF/bzvnd+r12GIMbwfdu3eDHIM/WatW/oS33fHXZCNuiQRen23kAfwxgJ8DMAbgswB+/Vac+2ZRn20cBvCbAP4BgFHQRf90fbZxAMB/AfAjACZAi+AXblc/Xwv12YYP4NcA5PjvgbkW9dmGA+BPADwD6usHAfxKfbZxCMDvAfgM5Nr8fn22sXPQ7C0Gz/2nAPzzWrVSBvAvAfxRfbZRwB18DeqzDa8+2/hxAL+N/poKO835HXU9thpDfbYxDLoe/4H7+TEAv1Wfbdw1SPeFxq0yobwbwEqtWvndWrXSA/DTAO6vzzbuuUXnvxkcAPDxWrXyeK1aiWvVyu8AiAF8GMCTtWrlK7VqpQPgpwD8rfpso3wb+7oTfhLA19Xfg3QtHgUwAuBf1qqVsFatfAvA2wAMATgC4D/UqpWgVq38GoA1AN93+7q6JSYBjAPw+GUEAB0A78KdfQ3+PYAf5E8AAPdtyznfadut73qKTWMAhTH/ea1a+T2+p78M0qzfgsG6L1Lcqgf4cQAnzR+1aiUCcBbAHTs5/ID+p+bv+mzjEdCDYxz9Y5kH0AIt4DsK9dnGmwH8EIB/oZoH6Vo8COAlAL9Yn23M1WcbLwI4DJrrV2rViiLbxmncYWOoVSvXQGaFTwMIAPw+gA/hzr8GP1+rVt4J6pPBcWw/5zttu13YNIZatfJSrVr5n83fbJ67D8CLuPOvyZa4VQ/wEoCNJdVbAIpb7HvHoT7bOALgD0Eq8ECMpT7byIJMJ/9brVrR/R2I/jPGALwH9DCYAfDjIPPV/RiAMbAJYQXAD4D69vcA/AYoMeOO7X+tWrm6RfNO6+aOW1PbjCFFfbYxBeBPAXyiVq28hDtwDDeCW/UAbwEobGgrAli/Ree/abBj4y8A/GqtWvk5DM5Y/hWAr9aqlb/Y0D4o/QeALoALtWrll2rVSq9WrXwOwBMgm+YgjOFvgBzH/437/zsAngPVeRmE/mvstG4GaU0Zh+s3ADwF4B9x80CNweBWPcBPgjy7AFLJ5DCUynIngqNNPg9yQv3f3LxxLBWQaeXM5iPcVvxNAB+pzzaW67MNU+K+DuAaBudanAIwouzHANU6XwFweIOT7BjuvDHsA1S9MEIA4DoG5xoYnMT2c77TtjsK9dnGWwA8CeC3atXKD9eqFVNvbSCfUbcqjPArACbqs40PA/hdAP8MwOlatXLHTg5Hm/wegA/XqpU/VJs+BeDf1Wcb7wHwOMjZ8elatXJHvalr1cpx/Xd9tpGAQrwuA/j5AbkWXwBJ4f+6Ptv4v0BOsccAfBTA3wfwz+uzjX8P4O+Cogq+enu6uS2+COCn67ONHwLwSQAfAPAIgP8VwL8akGsAAKhVK9+pzzbOYYs5r1Urne223abubgkONPg0gJ+pVSsbI8cG7hkF3CIJnG2w7wfwD0GxvN8LCgm7k/FPQHax36jPNtbNf9Bb+oOgUKkGgD2gG3IgMEjXolattEDRAY+C+vqfAPzdWrVyHsBf523XAfzvAH6Qo4LuGNSqlRcA/B2QE3kZwP8J6ud5DMg12ICd5vyOvx6gl/5uAP9G39P12caHBum+0LBkVhYWFhYDCptKb2FhYTGgsA9wCwsLiwGFfYBbWFhYDCjsA9zCwsJiQGEf4BYWFhYDCvsAt7CwsBhQ2Ae4hYWFxYDCPsAtLCwsBhT2AW5hYWExoPj/ATPuAiy32mRmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "dark"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GroundTruth:    cat  ship  ship plane\n"
     ]
    }
   ],
   "source": [
    "dataiter = iter(testloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# print images\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ok，现在让我们看看神经网络认为上面的例子是:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "outputs = net(images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "输出是10个类别的量值。一个类的值越高，网络就越认为这个图像属于这个特定的类。让我们得到最高量值的下标/索引；"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted:    cat   car plane plane\n"
     ]
    }
   ],
   "source": [
    "_, predicted = torch.max(outputs, 1)\n",
    "\n",
    "print('Predicted: ', ' '.join('%5s' % classes[predicted[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结果还不错。\n",
    "\n",
    "让我们看看网络在整个数据集上表现的怎么样。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the network on the 10000 test images: 56 %\n"
     ]
    }
   ],
   "source": [
    "correct = 0\n",
    "total = 0\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        total += labels.size(0)\n",
    "        correct += (predicted == labels).sum().item()\n",
    "\n",
    "print('Accuracy of the network on the 10000 test images: %d %%' % (\n",
    "    100 * correct / total))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这比随机选取（即从10个类中随机选择一个类，正确率是10%）要好很多。看来网络确实学到了一些东西。\n",
    "\n",
    "那么哪些是表现好的类呢？哪些是表现的差的类呢？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of plane : 65 %\n",
      "Accuracy of   car : 74 %\n",
      "Accuracy of  bird : 30 %\n",
      "Accuracy of   cat : 44 %\n",
      "Accuracy of  deer : 46 %\n",
      "Accuracy of   dog : 53 %\n",
      "Accuracy of  frog : 68 %\n",
      "Accuracy of horse : 64 %\n",
      "Accuracy of  ship : 49 %\n",
      "Accuracy of truck : 68 %\n"
     ]
    }
   ],
   "source": [
    "class_correct = list(0. for i in range(10))\n",
    "class_total = list(0. for i in range(10))\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs, 1)\n",
    "        c = (predicted == labels).squeeze()\n",
    "        for i in range(4):\n",
    "            label = labels[i]\n",
    "            class_correct[label] += c[i].item()\n",
    "            class_total[label] += 1\n",
    "\n",
    "\n",
    "for i in range(10):\n",
    "    print('Accuracy of %5s : %2d %%' % (\n",
    "        classes[i], 100 * class_correct[i] / class_total[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ok，接下来呢？\n",
    "\n",
    "怎么在GPU上运行神经网络呢？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 在GPU上训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与将一个张量传递给GPU一样，可以这样将神经网络转移到GPU上。 \n",
    "\n",
    "如果我们有cuda可用的话，让我们首先定义第一个设备为可见cuda设备："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda:0\n"
     ]
    }
   ],
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "# Assuming that we are on a CUDA machine, this should print a CUDA device:\n",
    "\n",
    "print(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本节的其余部分假设`device`是CUDA。\n",
    "\n",
    "然后这些方法将递归遍历所有模块，并将它们的参数和缓冲区转换为CUDA张量："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Net(\n",
       "  (conv1): Conv2d(3, 6, kernel_size=(5, 5), stride=(1, 1))\n",
       "  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
       "  (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))\n",
       "  (fc1): Linear(in_features=400, out_features=120, bias=True)\n",
       "  (fc2): Linear(in_features=120, out_features=84, bias=True)\n",
       "  (fc3): Linear(in_features=84, out_features=10, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "net.to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请记住，我们不得不将输入和目标在每一步都送入GPU："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "inputs, labels = inputs.to(device), labels.to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为什么我们感受不到与CPU相比的巨大加速？因为我们的网络实在是太小了。\n",
    "\n",
    "尝试一下：加宽你的网络（注意第一个`nn.Conv2d`的第二个参数和第二个`nn.Conv2d`的第一个参数要相同），看看能获得多少加速。\n",
    "\n",
    "已实现的目标：\n",
    "\n",
    "* 在更高层次上理解PyTorch的Tensor库和神经网络\n",
    "* 训练一个小的神经网络做图片分类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 在多GPU上训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果希望使用您所有GPU获得**更大的**加速，请查看[Optional: Data Parallelism](https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 接下来要做什么？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* [Train neural nets to play video games](https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html)\n",
    "* [Train a state-of-the-art ResNet network on imagenet](https://github.com/pytorch/examples/tree/master/imagenet)\n",
    "* [Train a face generator using Generative Adversarial Networks](https://github.com/pytorch/examples/tree/master/dcgan)\n",
    "* [Train a word-level language model using Recurrent LSTM networks](https://github.com/pytorch/examples/tree/master/word_language_model)\n",
    "* [More examples](https://github.com/pytorch/examples)\n",
    "* [More tutorials](https://github.com/pytorch/tutorials)\n",
    "* [Discuss PyTorch on the Forums](https://discuss.pytorch.org/)\n",
    "* [Chat with other users on Slack](https://pytorch.slack.com/messages/beginner/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "220.797px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
